ITERATIVE TECHNIQUES FOR VIDEO CONTENT ENCODING

专利摘要:
in various modalities, an iterative encoding application encodes a source video stream. the coding optimization application generates a set of trigger coding points based on a set of coding points and a first trigger sequence included in the source video sequence. each trigger coding point is associated with a different coded trigger sequence. the coding optimization application performs convex wrap operations through the set of trigger coding points to generate a first convex wrap associated with the first trigger sequence. subsequently, the encoding optimization application generates encoded video sequences based on the first convex envelope and a second convex envelope associated with a second trigger sequence included in the source video sequence. the encoding optimization application calculates a new encoding point based on the encoded video streams and a target value for a first video metric, and then generates an optimized encoded video sequence based on the new encoding point.
公开号:BR112019017608A2
申请号:R112019017608-0
申请日:2018-02-23
公开日:2020-03-24
发明作者:Katsavounidis Ioannis
申请人:Netflix, Inc.；
IPC主号:

专利说明:

ITERATIVE TECHNIQUES FOR VIDEO CONTENT ENCODING
CROSS REFERENCE FOR RELATED APPLICATIONS [001] This application claims priority benefit from United States Provisional Patent Application having Serial Number 62 / 462,873 (Attorney Document Number NETF / 0144USL) and filed on February 23, 2017, the United States Provisional Patent Application having Serial Number 62 / 534,170 (Attorney Document Number NETF / 0176USL) and filed on July 18, 2017, United States Provisional Patent Application having Serial Number 62 / 550,517 ( Attorney Document NETF / 0176USL02) and filed on August 25, 2017, and the United States patent application having Serial Number 15 / 902,976 (Attorney Document Number NETF0144US4) and filed on February 22, 2018. The subject of these related orders is hereby incorporated by reference.
FUNDAMENTALS
FIELD OF VARIOUS MODALITIES [002] The modalities of the present invention generally refer to video technology and, more specifically, to iterative techniques for encoding video content.
DESCRIPTION OF RELATED TECHNIQUE [003] A typical video streaming service provides access to a library of media titles that can be viewed on a variety of different terminal devices. Each terminal device can connect to the video streaming service under different connection conditions. Some notable connection conditions include, without limitation, bandwidth and latency. In addition, each different terminal device can include
Petition 870190082340, of 08/23/2019, p. 12/28
2/68 different hardware for outputting the media title to the end user (s). For example, a particular terminal device could include a display screen with a specific screen size and a specific screen resolution.
[004] In many implementations, a terminal device that connects to a streaming video service runs a terminal application that determines, for a given media title, an appropriate encoded version of the media title to transmit to the device based on connection conditions and the properties of the terminal device. More specifically, the terminal application attempts to select a particular encoded version of the media title that provides the best visual quality when playing the media title on the terminal device, avoiding playback interruptions due to buffering or re-buffering .
[005] In some implementations, the terminal application selects the particular encoded version of the media title based on a bit rate ladder. The bit rate ladder is designed to achieve a target visual quality when playing a media title based on an available bit rate. Each line in the bit rate ladder specifies a different bit-resolution pair corresponding to a different pre-generated encoded version of the media title. To generate the encoded version of a media title corresponding to a given bit-resolution pair, the video service provider samples the video content associated with the media title in the resolution
Petition 870190082340, of 08/23/2019, p. 12/29
3/68 to generate sampled video content. The video service provider then encodes the sampled video content based on the encoding parameters associated with the bit rate.
[006] A disadvantage of the above encoding technique is that the complexity of the video content associated with a given media title often varies over the media title, but the resolution and encoding parameters used to encode the video content do not vary. As a result, encoding relatively simple parts of the media title can consume more computational and storage resources than necessary to meet the desired visual quality. For example, a relatively simple piece of a media title can have the same visual quality, regardless of whether that piece of video content is encoded using a bit rate of 8 megabits per second or encoded using a bit rate of 20 megabits per second. Such coding inefficiencies not only waste computer and storage resources unnecessarily, these types of inefficiencies also unnecessarily increase the bandwidth needed to transmit coded versions of media titles to terminal devices.
[007] As the previous one illustrates, what is needed in the technique are more effective techniques for encoding video content for continuous transmission.
SUMMARY [008] One embodiment of the present invention establishes a computer-implemented method for encoding source video sequences. The method includes generating a first
Petition 870190082340, of 08/23/2019, p. 12/30
4/68 set of shooting encoding points based on a first set of encoding points and a first shooting sequence included in a source video sequence that is associated with a media title, where each shooting encoding point it is associated with a different coded firing sequence; performing one or more convex wrap operations through the first set of trigger coding points to generate a first convex wrap that is associated with the first trigger sequence; generating encoded video sequences based on the first convex envelope and a second convex envelope that is associated with a second trigger sequence included in the source video sequence; calculate a first encoding point that is not included in the first set of encoding points based on the encoded video streams and a target value for a first video metric; and generating an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently transmitted continuously to a terminal device during playback of the media title.
[009] At least one technological improvement of the techniques disclosed in relation to the prior art is that, iteratively converging to an optimized set of individually coded firing sequences reduces the coding inefficiencies typically associated with conventional coding techniques. In particular, because each trigger sequence is encoded based on a resolution and an encoding parameter (s) that is optimized
Petition 870190082340, of 08/23/2019, p. 12/31
5/68 for the trigger sequence, the encoded video sequence can be transmitted continuously to terminal devices with improved visual quality for a target bit rate. On the other hand, the encoded video sequence can be transmitted continuously to terminal devices with a reduced bit rate for a target visual quality.
BRIEF DESCRIPTION OF THE DRAWINGS [0010] So that the way in which the resources cited above of the various modalities can be understood in detail, a more particular description of the inventive concepts, summarized above, can be made by reference to various modalities, some of which are illustrated in the accompanying drawings. It should be noted, however, that the attached drawings illustrate only typical modalities of the inventive concepts and, therefore, they should not be considered in any way as limiting the scope and that there are other equally effective modalities.
[0011] Figure 1 is a conceptual illustration of a system configured to implement one or more aspects of the present invention;
Figure 2 is a more detailed illustration of the iteration controller of Figure 1, according to various embodiments of the present invention;
Figure 3 is a more detailed illustration of the dynamic optimizer of Figure 1, according to various modalities of the present invention;
Figure 4 illustrates the convex envelope that is generated by the convex envelope generator of Figure 3, according to various embodiments of the present invention;
Petition 870190082340, of 08/23/2019, p. 12/32
6/68
Figures 5A-5D illustrate in greater detail how the lattice iterator of Figure 3 assembles encoded shooting sequences into encoded video sequences, according to various embodiments of the present invention;
Figure 6 illustrates the global convex envelope that is generated by the dynamic optimizer of Figure 1 based on the different video coding points shown in Figures 5A-5D, in accordance with various modalities of the present invention;
Figure 7 illustrates how the iteration controller in Figure 1 generates additional coding points for a trigger sequence, in accordance with various embodiments of the present invention;
Figure 8 is a more detailed illustration of the optimized encoded video sequence of Figure 1, according to various embodiments of the present invention; and
Figures 9A-9B establish a flow chart of method steps for encoding a source video sequence, in accordance with various embodiments of the present invention.
DETAILED DESCRIPTION [0012] In the following description, numerous specific details are set out to provide a more complete understanding of the various modalities. However, it will be apparent to a person skilled in the art that inventive concepts can be practiced without one or more of these specific details.
[0013] The disclosed techniques generate an encoded video sequence optimized based on a target metric value (for example, a bit rate or a quality score). For each capture sequence included in a
Petition 870190082340, of 08/23/2019, p. 12/33
7/68 source video sequence, an iteration controller initializes a coding list to include coding points with relatively dispersed distribution. Each coding point specifies a resolution and a quantification parameter (QP). The iteration controller then sets up a dynamic optimizer to generate a global convex envelope based on the coding lists.
[0014] The dynamic optimizer generates new trigger coding points based on the new coding points included in the coding lists. A new coding point is a coding point for which the dynamic optimizer has not previously generated an associated coded firing sequence. As part of generating a given trigger encoding point, the dynamic optimizer generates and analyzes an associated encoded capture sequence. Subsequently, for each trigger sequence, the dynamic optimizer generates a convex envelope that includes a subset of the trigger coding points associated with the trigger sequence. In general, for a given trigger sequence, the trigger encoding points included in the associated convex envelope minimize the bit rate for different levels of distortion.
[0015] The dynamic optimizer then evaluates the convex wraps across all the trigger sequences to identify multiple trigger coding sequences, where each trigger coding sequence specifies a sequence of trigger coding points. For each trigger coding sequence, the dynamic optimizer aggregates the different coded trigger sequences associated with the trigger coding points
Petition 870190082340, of 08/23/2019, p. 12/34
8/68 to generate an encoded video stream. The dynamic optimizer generates a global convex envelope based on the trigger coding sequences. In general, each point included in the global convex envelope is associated with a different trigger encoding sequence that minimizes the overall bit rate for a different level of overall distortion.
[0016] The iteration controller selects an optimized trigger encoding sequence based on the global convex envelope and a target metric value (for example, bit rate or quality score). For each trigger sequence, the optimized trigger coding sequence specifies an optimized trigger coding point. For each trigger sequence, the iteration controller evaluates the location of the optimized trigger coding point along the associated convex envelope to identify one or more nearby trigger codes. Based on the optimized trigger coding point resolutions and QP values and the nearby trigger coding points, the iteration controller can generate any number of new coding points for the trigger sequence. If the iteration controller identifies any new coding points, then the iteration controller configures the dynamic optimizer to generate a new global convex envelope based on the expanded set of coding points.
[0017] If, however, the iteration controller does not identify any new encoding points, then the iteration controller selects the encoded video sequence included in the video encoding point
Petition 870190082340, of 08/23/2019, p. 12/35
9/68 optimized as the optimized encoded video stream. Finally, the iteration controller transmits the optimized encoded video stream to a content delivery network for distribution to terminal devices.
[0018] An advantage and technological advancement of the described techniques is that the portions of the source video sequence needing specific encoding points to satisfy a target bit rate or the target quality score are encoded in the encoding points. In addition, other portions of the source video sequence are encoded at other appropriately chosen encoding points. Coding each trigger sequence at coding points that are optimized for the trigger sequence reduces the coding inefficiencies typically associated with conventional coding techniques.
SYSTEM OVERVIEW [0019] Figure 1 is a conceptual illustration of a system 100 configured to implement one or more aspects of the present invention. As shown, system 100 includes, without limitation, any number of computing instances 110 and a cloud 160. In general, cloud 160 contains encapsulated shared resources, software, data, etc. For explanatory purposes, several instances of similar objects are denoted with reference numbers that identify the object and numbers in parentheses that identify the instance when necessary.
[0020] Any number of computing instances 110 may reside outside the cloud 160 while other computing instances 110 may reside within the cloud.
Petition 870190082340, of 08/23/2019, p. 12/36
10/68 cloud 160. In various modalities, any number of components of system 100 can be distributed across various geographical locations. In addition, any number of system components 100 can be included or distributed across one or more independent devices, distributed computing environments, or clouds 160 in any combination.
[0021] As shown, each of the computing instances 110 includes, without limitation, a processor 112 and a memory 116. Processor 112 can be any instruction execution system, apparatus or device capable of executing instructions. For example, processor 112 may comprise a central processing unit (CPU), a graphics processing unit (GPU), a controller, a microcontroller, a state machine or any combination thereof. Memory 116 stores content, such as software applications and data, for use by processor 112 of computing instance 110.
[0022] Memory 116 may be one or more of readily available memory, such as random access memory (RAM), read-only memory (ROM), floppy disk, hard disk or any other form of digital storage, local or remote . In some embodiments, a storage (not shown) can supplement or replace memory 116. The storage can include any number and type of external memories that are accessible to the 112 processor. For example, and without limitation, the storage can include a Digital Card Secure, an external Flash memory, a read-only portable compact disc (CD-ROM) memory, a
Petition 870190082340, of 08/23/2019, p. 37/125
11/68 optical storage, a magnetic storage device or any suitable combination of the above.
[0023] In general, computing instances 110 are configured to implement one or more applications. For explanatory purposes only, each application is represented as residing in memory 116 of a single computing instance 110 and running on a processor 112 of the single computing instance 110. However, as those skilled in the art will recognize, the functionality of each application can be distributed through any number of other applications residing in memories 116 of any number of computing instances 110 and running on processors 112 from any number of computing instances 110 in any combination. In addition, the functionality of any number of applications can be consolidated into a single application or subsystem.
In particular, computing instances 110 are configured to generate an optimized encoded video sequence 180 for source video sequence 122. Source video sequence 122 includes, without limitation, any quantity and type of content from video associated with a media title. Examples of video content include, without limitation, any part (including all) of feature films, episodes of television shows and music videos, to name a few. The optimized encoded video sequence 180 includes, without limitation, encoded video content derived from the video content included in the source video sequence 122.
[0025] Although not shown, a streaming video service provides access to a library
Petition 870190082340, of 08/23/2019, p. 38/125
12/68 of media titles that can be viewed on a variety of different terminal devices. The media title library includes, without limitation, the media title associated with the source video stream 122. Each terminal device can connect to the video streaming service under different connection conditions. Some notable connection conditions include, without limitation, bandwidth and latency. In addition, each different terminal device may include different hardware for outputting the media title to the end user (s). For example, a particular terminal device could include a display screen with a specific screen size and a specific screen resolution.
[0026] In many implementations, a terminal device that connects to a streaming video service runs a terminal application that determines, for a given media title, an appropriate encoded version of the media title to transmit to the device based on connection conditions and the properties of the terminal device. More specifically, the terminal application attempts to select a particular encoded version of the media title that provides the best visual quality when playing the media title on the terminal device, avoiding playback interruptions due to buffering or re-buffering .
[0027] In some conventional implementations, the terminal application selects the particular encoded version of the media title based on a rate ladder
Petition 870190082340, of 08/23/2019, p. 39/125
13/68 bits. The bit rate ladder is designed to achieve a target visual quality when playing a media title based on an available bit rate. Each step in the bit rate ladder specifies a different bit-resolution pair corresponding to a different pre-generated encoded version of the media title. To generate the encoded version of a media title corresponding to a given resolution bit rate pair, the video service provider samples the video content associated with the media title in the resolution to generate sampled video content. The video service provider then encodes the sampled video content based on the encoding parameters associated with the bit rate.
[0028] A disadvantage of the conventional encoding technique above is that the complexity of the video content associated with a given media title often varies over the media title, but the resolution and encoding parameters used to encode the video content do not vary. As a result, encoding relatively simple parts of the media title can consume more computational and storage resources than necessary to meet the desired visual quality. For example, a relatively simple piece of a media title can have the same visual quality, regardless of whether that piece of video content is encoded using a bit rate of 8 megabits per second or encoded using a bit rate of 20 megabits per second. Such coding inefficiencies not only waste computer and storage resources unnecessarily, these types of inefficiencies also increase
Petition 870190082340, of 08/23/2019, p. 40/125
14/68 unnecessarily the bandwidth needed to transmit encrypted versions of media titles to terminal devices.
OPTIMIZING CODING OPERATIONS FOR INDIVIDUAL SHOOTING SEQUENCES [0029] To solve the above problems, system 100 includes, without limitation, an iterative encoding application 120 that optimizes the resolution and encoding parameters for each shooting sequence 132 included in the sequence of source video 122. The source video sequence 122 includes, without limitation, any number of continuous, non-overlapping shooting sequences 132. Each of the shooting sequences 132 includes a set of frames that normally have similar temporal spatial properties and run for an uninterrupted period of time. The iterative encoding application 120 resides in one of the memories 116 and runs on one of the processors 112.
[0030] As shown, the iterative encoding application 120 includes, without limitation, a trigger analyzer 130, an iteration controller 140, and a dynamic optimizer 150. Upon receiving source video sequence 122, trigger analyzer 130 determines one or more trigger changes 172 included in the source video sequence 122. Each trigger change 172 specifies a boundary between a different pair of trigger sequences 132. The trigger analyzer 130 can determine one or more trigger changes 172 in any technically feasible way.
[0031] For example, in some modalities, the
Petition 870190082340, of 08/23/2019, p. 41/125
15/68 trigger analyzer 130 transmits the source video sequence 122 to a trigger detector 170 that resides in the cloud 160. To determine trigger changes 172, trigger detector 170 performs any number of trigger detection algorithms with based on source video sequence 122. Some examples of trigger detection algorithms include, without limitation, a multi-scale absolute difference sum algorithm, a motion compensated residual energy algorithm, a difference histogram algorithm, a histogram difference algorithm and so on. The trigger detector 170 then transmits the trigger changes 172 to the trigger analyzer 130. In alternative embodiments, the trigger analyzer 130 can perform any number of trigger detection operations in the source video sequence 122 to determine the trigger changes 172.
[0032] The trigger analyzer 130 performs partitioning operations on the source video sequence 122 based on trigger changes 172 to determine the trigger sequences 132. In some embodiments, the trigger analyzer 130 can also remove extraneous pixels from the sequence source video 122. For example, the trigger analyzer 130 could remove pixels enclosed in black bars along edge sections of the source video sequence 122.
[0033] In various embodiments, the iterative encoding application 120 ensures that the initial frame of each trigger sequence 132 is encoded as a key frame during encoding operations. In general, a key frame and all subsequent frames of the same
Petition 870190082340, of 08/23/2019, p. 42/125
16/68 shooting sequence 132, which are included in an encoded video sequence, are decoded independently of any preceding frames included in the encoded video sequence.
[0034] The iterative encoding application 120 can ensure that the different initial frames of the different trigger sequences 132 are encoded as key frames in any technically feasible manner. For example, in some embodiments, the dynamic optimizer 150 sets up an encoding application 160 to encode frames as keyframes based on a keyframe location list (not shown in Figure 1) when encoding video content. In other embodiments, the dynamic optimizer 150 can perform any number of encoding operations to encode the different starting frames of the different shooting sequences 132 as key frames when encoding video content.
[0035] As technicians in the subject will recognize, during playback, the media title associated with the source video sequence 122 can be switched between decoded versions of different 180 encoded video sequences in aligned key frames to optimize a viewing experience based on any number of relevant criteria. Examples of relevant criteria include the current connection bandwidth, the current connection latency, the content of the next trigger sequence 132 and the like.
[0036] Upon receiving trigger sequences 132, iteration controller 140 generates an optimized encoded video sequence 180 based on a metric value
Petition 870190082340, of 08/23/2019, p. 43/125
17/68 target 170. For each of the 132 shooting sequences, the optimized encoded video sequence 180 includes, without limitation, an encoded shooting sequence (not shown in Figure 1) that is associated with the shooting sequence 132. Each of the encoded shooting sequences includes encoded video content derived from the video content included in the associated shooting sequence 132.
[0037] The target metric value 170 is a value for a metric that corresponds to one or more properties of encoded video content. In some embodiments, the target metric value 170 is a bit rate. In other modalities, the target metric value 170 is a quality score of a visual quality metric. Examples of visual quality metrics include, without limitation, a peak signal-to-noise ratio (PSNR), a linear video multi-method evaluation fusion metric (VMAF), and a harmonic VMAF (VMAFh), to name a few.
[0038] In operation, iteration controller 140 generates the optimized encoded video sequence 180 in an iterative process that individually optimizes each of the encoded shooting sequences included in the optimized encoded video sequence 180. As described in greater detail together with Figure 2, for each trigger sequence 132, iteration controller 140 initializes a different coding list (not shown in Figure 1) to include coding points with relatively dispersed distribution. In general, each coding list can specify any number of coding points and the number of coding points in the coding list may differ from the number of coding points in others
Petition 870190082340, of 08/23/2019, p. 44/125
18/68 coding lists. Each encoding point specifies a resolution and one or more encoding parameters.
[0039] Subsequently, as described in greater detail in conjunction with Figures 3-6, the iteration controller 140 configures the dynamic optimizer 150 to generate a global convex envelope based on the coding lists. First, the dynamic optimizer 150 evaluates the coding lists to identify new coding points. For each coding list, a new coding point is a coding point for which the dynamic optimizer 150 has not previously generated a coded firing sequence based on the associated firing sequence 132. For each new coding point, the dynamic optimizer 150 generates an encoded firing sequence.
[0040] The dynamic optimizer 150 can generate the encoded firing sequences in any technically feasible way. In addition, as a general matter, the dynamic optimizer 150 can generate encoded video content derived from video content based on a given resolution and data encoding parameters in any technically feasible manner. For example, in some embodiments, the dynamic optimizer 150 performs sampling operations on the video content based on the resolution to generate sampled video content. Subsequently, the dynamic optimizer 150 configures the encoding application 162 to perform encoding operations on the sampled video content based on the encoding parameters to generate the encoded shooting sequence.
Petition 870190082340, of 08/23/2019, p. 45/125
19/68 [0041] As shown, encoding application 162 resides in cloud 160 and is configured to efficiently perform encoding operations through one or more parallel encoders 164. Each of the parallel encoders 164 can include any number of computer instances 110. In alternative modalities, dynamic optimizer 150 can perform coding operations and system 100 can omit coding application 162. In the same or other modalities, system 100 can include a sampling application, and dynamic optimizer 150 can include configure the sampling application to perform sampling operations.
[0042] The dynamic optimizer 150 can perform sampling and coding operations at any level of granularity (for example, by frame, by shooting sequence, by video sequence, etc.) in any combination and in any technically feasible way. For example, in some embodiments, the dynamic optimizer 150 can perform sampling operations on the source video stream 122 based on the resolution given to generate a sampled video stream. Subsequently, for each encoding point associated with the resolution, the dynamic optimizer 150 can configure the encoding application 162 to encode the sampled video content corresponding to the associated trigger sequence 132 based on the associated encoding parameters.
[0043] After generating a new encoded trigger sequence, the dynamic optimizer 150 calculates a bit rate, a quality score, and a level of distortion based on the encoded trigger sequence. The optimizer
Petition 870190082340, of 08/23/2019, p. 46/125
Dynamic 20/68 150 can calculate the bit rate, quality score and level of distortion in any technically feasible way. For each new encoded trigger sequence, dynamic optimizer 150 generates a trigger encoding point (not shown in Figure 1) that includes the encoded trigger sequence, resolution, encoding parameters, bit rate, quality, and the level of distortion. For each of the firing sequences 132, the dynamic optimizer 150 generates a different convex envelope. The convex envelope associated with a given firing sequence 132 includes any number of the firing coding points associated with firing sequence 132. In general, for a given firing sequence 132, the firing encoding points included in the associated convex envelope minimize the bit rate for different levels of distortion.
[0044] The dynamic optimizer 150 then evaluates the convex wraps across all of the trigger sequences 132 to determine trigger encoding sequences (not shown in Figure 1). Each firing coding sequence specifies firing coding points for the different firing sequences 132. For each firing coding sequence, the dynamic optimizer 150 aggregates the different coded firing sequences included in the firing coding points to generate a sequence encoded video. For each trigger encoding sequence, dynamic optimizer 150 then generates a video encoding point that includes the trigger encoding sequence, the associated encoded video sequence, an overall bit rate for the
Petition 870190082340, of 08/23/2019, p. 47/125
21/68 encoded trigger sequence, and an overall distortion level for the encoded trigger sequence. Subsequently, the dynamic optimizer 150 generates a global convex envelope based on the video encoding points. In general, for source video sequence 122, each of the video encoding points included in the global convex envelope minimizes the overall bit rate to a different level of global distortion.
[0045] Iteration controller 140 then selects an optimized video encoding point that is included in the global convex wrapper based on target metric value 170. For each trigger sequence 132, iteration controller 140 identifies a point optimized trigger encoding based on the video encoding point. Subsequently, for each firing sequence 132, iteration controller 140 identifies any number (including zero) of new coding points based on the associated optimized firing coding point and the convex envelope associated with firing sequence 132 (x).
[0046] More specifically, for a given trigger sequence 132, iteration controller 140 performs any number and type of search operations on the convex envelope associated with trigger sequence 132 based on the optimized trigger coding point associated with the sequence trigger 132. If iteration controller 140 identifies any new coding points, then iteration controller 140 adds the coding point (s) to the appropriate coding list. The iteration controller 140 then configures the dynamic optimizer 150
Petition 870190082340, of 08/23/2019, p. 48/125
22/68 to generate a new global convex envelope based on the expanded coding lists.
[0047] If, however, iteration controller 140 does not identify any new coding points, then iteration controller 140 determines that iteration controller 140 has converged successively with target metric value 170. Iteration controller 140 selects the encoded video sequence included in the optimized video encoding point as the optimized encoded video sequence 180. Finally, iteration controller 140 transmits the optimized encoded video sequence 180 to a content delivery network (CDN) 190 for distribution to terminal devices.
[0048] Note that the techniques described here are illustrative rather than restrictive and can be changed without departing from the broader spirit and scope of the invention. Many modifications and variations in functionality provided by the iterative coding application 120, the trigger analyzer 130, the iteration controller 140, the dynamic optimizer 150, the trigger detector 170, the coding application 162, and the content delivery network 190 will be apparent to those with current knowledge of the technique without departing from the scope and spirit of the modalities described. For example, in several embodiments, one or more of the iterative coding application 120, the trigger analyzer 130, the iteration controller 140, the dynamic optimizer 150 and the coding application 162 can be configured to identify and operate on framesets for which a consistency metric is within a specified range instead of firing strings
Petition 870190082340, of 08/23/2019, p. 49/125
23/68
132.
[0049] In some alternative modalities, the application of iterative coding 120, the trigger analyzer 130, and / or the trigger detector 170 can be configured to partition the source video sequence 132 into sets of frames that are not necessarily equivalent to the shooting sequences 132. The iterative encoding application 120, the shooting analyzer 130 and / or the shooting detector 170 can be configured to divide the source video sequence 132 into the framesets in any technically feasible manner. A set of frames can represent a variety of different constructions, including a group of images (GOP), a frame sequence, a plurality of frame sequences, and so on. In addition, the iterative encoding application 120, the iteration controller 140, the dynamic optimizer 150, the encoding application 162 and the content delivery network 190 can be configured to operate on framesets and framesets encoded in instead of triggering sequences 132 and coded triggering sequences.
[0050] As those skilled in the art will recognize, the techniques described here can be modified to optimize audio encoding instead of video encoding. For example, in some modalities, an audio track can be partitioned into audio scenes. Audio scenes can be sampled using audio rendering hardware. The sampled audio scenes can be encoded using an audio codec that is configured using a quantization parameter and / or rate settings.
Petition 870190082340, of 08/23/2019, p. 50/125
24/68 bits. The quality scores of the encoded audio scenes can be calculated using a Perceptual Audio Quality Assessment (PEAQ) algorithm. Notably, the resolution and / or any number of encoding parameters can be optimized for each audio scene based on any of the techniques described here in any combination.
[0051] It should be noted that the system 100 shown here is illustrative and that variations and modifications are possible. For example, the functionality provided by the iterative coding application 120, the trigger analyzer 130, the iteration controller 140, the dynamic optimizer 150, the trigger detector 170, the coding application 162 and the content delivery network 190 as described here can be integrated or distributed in any number of software applications (including one), hardware devices (for example, a hardware based encoder) and any number of system components 100. In addition, the connection topology between several units in Figure 1 can be modified as desired.
[0052] Figure 2 is a more detailed illustration of the iteration controller 140 of Figure 1, according to various embodiments of the present invention. As shown, iteration controller 140 includes, without limitation, S + 1 coding lists 210, where S + 1 is the total number of trigger sequences 132. In addition, each coding list 210 includes any number of points code 120. Each coding point 120 includes, without limitation, a bit rate 332 and a parameter value of
Petition 870190082340, of 08/23/2019, p. 51/125
25/68 quantification (QP) 324.
[0053] The quantization parameter allows monotonic performance in terms of bit rate and distortion when encoding video content. The higher the QP value, the lower the resulting bit rate at the expense of lower quality. However, in alternative embodiments, each of the coding points 120 may include any number and type of coding parameters instead of or in addition to the value of QP 324.
[0054] Initially, for each trigger sequence 132 (x), iteration controller 140 generates coding list 210 (x) which includes a relatively scattered selection of coding points 220. Iteration controller 140 can generate lists initial coding codes 210 in any technically feasible form. For example, in some embodiments, iteration controller 140 initializes coding lists 210 based on a fixed set of resolutions and the codec implemented by the iterative coding application 120.
[0055] More specifically, for each coding list 210, the iteration controller 210 generates the coding point 220 (0) specifying a minimum resolution and the maximum QP value allowed by the video codex. The iteration controller 210 then generates the coding point 220 (1) specifying the minimum resolution and the average QP value allowed by the video codex. For each additional resolution that lies between the maximum resolution and the minimum resolution, iteration controller 210 generates coding point 220 specifying the resolution and the average QP value. The iteration controller 210 then generates the
Petition 870190082340, of 08/23/2019, p. 52/125
26/68 coding 220 specifying the maximum resolution and the average QP value. Finally, iteration controller 210 generates encoding point 220 specifying the maximum resolution and minimum QP value allowed by the video codex.
[0056] For explanatory purposes only, examples of resolutions and values for the first five points included in each of the 210 coding lists are represented in italics. As shown, coding point 220 (0) specifies the resolution 322 of 640x260 and the QP 324 value of 1920 x 1080 and the QP value of 324 is 0, coding point 220 (1) specifies the resolution 322 of 640x260 and the QP value 324 of 26, the coding point 220 (2) specifies the 322 resolution of 960x520 and the QP value 324 of 26, the coding point 220 (3) specifies the 322 resolution of 1280x720 and the value of QP 324 of 26, coding point 220 (4) specifies the 322 resolution of 1920 x 1080 and the QP value 324 of 26, and coding point 220 (5) specifies the 322 resolution of 1920x1080 and the QP value 324 of 0.
[0057] As described in greater detail in conjunction with Figure 7, after the dynamic optimizer 150 generates a new global convex wrapper, iteration controller 140 can update any number of coding lists 120. Notably, iteration controller 140 independently evaluates each trigger sequence 132 (x) to determine any number of new coding points 220 to add to coding lists 210 (x). As a result, at any one time, the number of coding points 220 included in coding list 210 (x) may vary from the number of coding points 220 included in any of the other coding lists 210. In
Petition 870190082340, of 08/23/2019, p. 53/125
In general, iteration controller 140 iteratively refines the range of encoding points 220 in order to efficiently converge to the optimized encoded video sequence 180 that best corresponds to the target metric value 170.
GENERATING DIFFERENT ENCODED VIDEO SEQUENCES [0058] Figure 3 is a more detailed illustration of the dynamic optimizer 150 of Figure 1, according to various modalities of the present invention. As shown, dynamic optimizer 150 includes, without limitation, trigger encoding sets 310, a convex wrap generator 340, convex wraps 350, a 360 lattice iterator, a 370 sequence lattice, any number of video encoding points 380, and a global convex envelope 390. The total number of the trigger coding sets 310 is equal to the number of firing strings 132. Similarly, the total number of convex envelopes 340 is equal to the number of firing sequences 132. In general, the firing sequence 132 (x) is associated with both the firing coding set 310 (x) and the convex envelope 340 (x).
[0059] Each of the firing coding sets 310 includes, without limitation, any number of firing coding points 320. The number of firing coding points 320 included in firing coding set 310 (x) may differ from number of trigger coding points 320 included in any of the other trigger coding sets 310. Each of the trigger coding points 320 includes, without limitation, resolution 322, the value of QP 324, a sequence
Petition 870190082340, of 08/23/2019, p. 54/125
28/68 encoded trigger 326, a bit rate 332, any number of quality scores 334 and a level of distortion 336. In alternative embodiments, trigger coding point 320 can include any number and type of encoding parameters in instead of or in excess of QP 324.
[0060] When the dynamic optimizer 150 initially receives the coding lists 210 (0) - 210 (S) from the iteration controller 140, the dynamic optimizer 150 generates the trigger coding sets 310 (0) - 310 (S). For each coding point 220 (y) included in each coding list 210 (x), the dynamic optimizer 150 generates a corresponding trigger coding point 320 (y) and adds the trigger coding point 320 (y) to the set trigger code 310 (x). More specifically, dynamic optimizer 150 generates a new encoded firing sequence 326 based on firing sequence 132 (x), resolution 322 and the QP value 324 specified in coding point 220 (y).
[0061] The dynamic optimizer 150 then calculates the bit rate 332, the quality score 334, and the level of distortion 336 associated with the encoded trigger sequence 326. The dynamic optimizer 150 then generates the encoding point of trigger 320 (y), which includes, without limitation, the encoded trigger sequence 326 as well as the resolution 322, the QP value 324, the bit rate 332, the quality score 334 and the level of distortion 336 associated with the sequence coded trigger rate 326. Finally, the dynamic optimizer 150 includes the new trigger coding point 320 (y) in the trigger coding set 310 (x).
Petition 870190082340, of 08/23/2019, p. 55/125
29/68 [0062] Subsequently, when receiving one or more updated coding lists 210, dynamic optimizer 150 identifies any new coding points 220. In operation, for coding list 210 (x), dynamic optimizer 150 determines which (if any) of coding points 220 are not represented by trigger coding set 310 (x). For each new coding point 220 (y) included in coding list 210 (x), dynamic optimizer 150 generates a corresponding trigger coding point 320 (y) and adds the new trigger coding point 320 (y) to the trigger coding set 310 (x).
[0063] In general, the dynamic optimizer 150 can generate the encoded trigger sequence 326 and determine the associated quality score 334, associated bit rate 332, and the associated distortion level 336 in any technically feasible manner. To generate the encoded trigger sequence 326, in some embodiments, the dynamic optimizer 150 performs sampling operations on the video content based on resolution 322 to generate sampled video content. Subsequently, the dynamic optimizer 150 configures the encoding application 162 to perform encoding operations on the sampled video content based on the QP 324 to generate the encoded trigger sequence 326. The dynamic optimizer 150 can generate the encoded trigger sequence 326 on the basis of in any technically viable coding algorithm (s) in any technically viable manner. Examples of encoding algorithms include advanced video encoding (AVC) and high efficiency video encoding (HEVC), to name a few.
Petition 870190082340, of 08/23/2019, p. 56/125
In some embodiments, to determine the quality score 334 associated with the encoded trigger sequence 326, the dynamic optimizer 150 decodes the encoded trigger sequence 326 to generate a decoded trigger sequence. The dynamic optimizer 150 then resamples (i.e., sample above or sample below) the decoded firing sequence to a target resolution to generate a reconstructed firing sequence that is relevant to the display characteristics of a class of terminal devices.
[0065] In alternative modalities, the dynamic optimizer 150 can calculate any number of quality scores 334 for any number or resolutions. For example, a particular video can be delivered at 3840 x 2160 resolution, but it must be consumed by a large number of displays at 1920 x 1080 resolution. Another class of terminal devices, for example, laptop computers, display the same video in 1280 x 720 resolution. However, another class of terminal devices, for example, tablet or smart phone devices, is expected to display the same video in 960 x 540 resolution. The dynamic optimizer 150 could above samples the decoded firing sequence for all of these target resolutions in order to assess quality, when considering one of these different classes of terminal devices, correspondingly.
[0066] The dynamic optimizer 150 then analyzes the reconstructed firing sequence to generate the quality score 334 for an objective quality metric (QM). For example, in some modalities, the dynamic optimizer
Petition 870190082340, of 08/23/2019, p. 57/125
31/68
150 implements a VMAF (or harmonic VMAF) algorithm to generate a VMAF score for each encoded trigger sequence 326 based on the associated reconstructed trigger sequence. While a multitude of video quality metrics, such as VMAF scores, can be calculated at different target resolutions, it should be made clear that when comparing qualities between 385 encoded shooting sequences associated with different 332 resolutions, applications need to use the same resolution target for resampling, after decoding. For example, in some embodiments, dynamic optimizer 150 resamples the firing sequence decoded to 1920x1080 to generate the reconstructed firing sequence for each 326 encoded firing sequence. Subsequently, the dynamic optimizer calculates the quality score 334 for the sequence trip code 326 based on the associated reconstructed trip sequence.
[0067] Dynamic optimizer 150 can generate bit rate 332 based on resolution 334 in any technically feasible manner. For example, in some embodiments, the dynamic optimizer 150 can divide the total number of bits required for resolution 334 by the length of the associated trigger sequence 132. In the same or other modalities, the dynamic optimizer 150 can calculate the level of distortion 336 based on quality score 334 and any technically feasible technique for converting quality to distortion. For example, dynamic optimizer 150 could invert quality score 334 to determine the level of distortion 336. In another example, dynamic optimizer 150 could subtract the score from
Petition 870190082340, of 08/23/2019, p. 58/125
32/68 quality 334 from a constant value to determine the level of distortion 336.
[0068] For each of the firing sequences 132 (x), the convex wrap generator 340 generates a wrap
convex 350 (X) with base at the coding set in firing 310 (X). Each one From 350 convex wraps (x) includes, without limitation, the trigger coding points
320 included in the trigger encoding set 310 (x) that maximize the bit rate for a given level of distortion. A detailed example of how the convex envelope generator 340 generates the convex envelope 350 (0) based on the trigger coding set 310 (0) is described in conjunction with Figure 4.
[0069] As shown, and as described in detail in conjunction with Figure 5, the lattice iterator 360 receives the convex wraps 350 and then iteratively updates a sequence lattice 370 to generate any number of video encoding points 380 The truss iterator 360 is a software module, and the 370 sequence truss is a data structure that is described in greater detail below in conjunction with Figures 5A-5D.
[0070] Each of the video encoding points 380 includes, without limitation, a shooting encoding sequence 382, an encoding video sequence 382, bit rate 332, and distortion level 334. The encoding sequence of trigger 382 includes, without limitation, S + 1 trigger coding points 320 - a different trigger coding point 320 for each of the shooting sequences 132. The encoded video sequence 382 includes, without limitation, the S + 1 sequences firing
Petition 870190082340, of 08/23/2019, p. 59/125
33/68 encoded 326 included in the S + 1 320 encoding trigger points included in the 382 trigger encoding sequence. The bit rate 332 and the distortion level 334 specify, respectively, an overall bit rate and a distortion level global for the encoded video sequence 283. As described in greater detail in conjunction with Figures 5A-5D, the lattice iterator 360 generates the video encoding points and subsequently generates the global convex wrap 390 based on the points 360 video encoding.
[0071] Figure 4 illustrates the convex envelope 350 (0) that is generated by the convex envelope generator 340 of Figure 3, according to various modalities of the present invention. In particular, the convex envelope generator 340 generates the convex envelope 350 (0) based on the trigger coding set 310 (0). As shown, a graph 400 includes, without limitation, a bit rate axis 410 and a distortion axis 420.
[0072] In operation, the convex wrap generator
340 distributes the trigger coding points 320 included in the trigger coding set 310 (0) in different subsets based on resolution 322. Subsequently, for each specific subset of resolution, the convex envelope generator 340 plots each of the coding points trigger 320 by locating bit rate 332 along bit rate axis 410 and distortion level 336 along a distortion axis 420 to generate a corresponding distortion curve 450. In this way, the convex envelope generator 340 generates the distortion curves 450 (0) - 450 (3), where each of the
Petition 870190082340, of 08/23/2019, p. 60/125
34/68 distortion 450 corresponds to a different resolution 322 and includes one or more trigger coding points 320.
[0073] After generating the distortion curves 450, the convex envelope generator 340 evaluates the trigger coding points 320 along the distortion curves 450 to determine the convex envelope 350 (x). More specifically, the convex envelope generator 340 identifies the trigger coding points 320 through all distortion curves 450 that form a boundary where all trigger coding points 320 reside on one side of the boundary (in this case, the right of the border) and are also such that the connection of any two consecutive identified firing coding points 320 with a straight line leaves all the remaining firing coding points 320 on the same side. The convex wrap 350 (0) includes the set of identified trigger coding points 320.
[0074] Those skilled in the art will understand that many techniques for generating convex wraps are well known in the field of mathematics, and all of these techniques can be applied to generate 350 convex wraps. In one embodiment, the 340 convex wrap generator applies machine learning techniques to estimate the trigger encoding points 320 included in the convex envelope 350 based on various parameters of the associated source video sequence 132. In this way, some of the calculations discussed so far can be simplified and / or avoided entirely .
[0075] Figures 5A-5D illustrate in greater detail how the 360 lattice iterator of Figure 3 assembles sequences
Petition 870190082340, of 08/23/2019, p. 61/125
35/68 encoded shooting streams 336 in 386 encoded video sequences, according to various embodiments of the present invention. As shown in Figures 5A-5D, the sequence lattice 370 includes, without limitation, a trigger axis 520 and the bit rate axis 410. The sequence lattice 370 also includes, without limitation, columns of trigger coding points. 320 included in the convex wrappers 350, where each column corresponds to a particular firing sequence 132. For example, the zero-th column included in the sequence truss 370 corresponds to the trigger coding points 320 included in the convex wrapping 350 (0). The trigger encoding points 320 included in any column are classified according to the rising bit rate 332 (and, by design, the descending distortion levels 336). The wrapping trigger encoding points 320 included in any column also guarantee to have negative slopes which - in magnitude - are decreasing as a function of the bit rate 332.
[0076] For convenience, the wrap trigger firing coding points 320 are indexed individually according to the following system. For a given envelope firing encoding point 320, the first number is an index of the firing sequence 132, and the second number is an index for the bit rate classification of those envelope firing encoding points 320. For example , the envelope firing coding point 320 00 corresponds to the zero th firing sequence 132 (0) and the zero-ordered bit rate 332. Similarly, the envelope firing coding point 320 43 corresponds to the fourth sequence firing range 332 (4) and the third
Petition 870190082340, of 08/23/2019, p. 62/125
36/68 bits ordered 332 (in this case the highest bit rate 332).
[0077] As previously described in conjunction with Figure 3, each wrapping trigger coding point 320 included within the sequence truss 370 includes a different coded trigger sequence 326. The truss iterator 360 generates the encoded video sequences 386 for combining these encoded trigger sequences 326. The truss iterator 360 implements the sequence truss 370 to iteratively perform this combining technique.
[0078] Each of Figures 5A-5D illustrates a different version of the 370 sequence truss generated by the truss iterator 360 in a different iteration. Figure 5A illustrates the 370 (0) sequence lattice in an initial state. Here, the lattice iterator 360 generates the trigger coding sequence 382 (0) which includes the wrap triggering coding points 320 00, 10, 20, 30 and 40. These initially selected wrap triggering coding points 320 they have the lowest bit rate 332 and the highest distortion levels 336 and therefore reside at the bottom of the respective columns.
[0079] The lattice iterator 360 generates the encoded video sequence 386 (0) based on the encoded shooting sequence 382 (0). More precisely, the lattice iterator 360 aggregates the encoded shooting sequences 326 sequentially included in the envelope firing encoding points 00, 10, 20, 30 and 40 to generate the encoded video sequence 386 (0). Subsequently, the lattice iterator 360 calculates the bit rate 332 and the level
Petition 870190082340, of 08/23/2019, p. 63/125
37/68 distortion 336 of the 386 (0) encoded video sequence. The lattice iterator 360 can calculate the bit rate 332 and the level of distortion 336 of the encoded video sequence 386 (0) in any technically feasible way. The lattice iterator 360 then generates the video encoding point 380 (0) which includes, without limitation, the encoded shooting sequence 382 (0), the encoded video sequence 386 (0), the bit rate 332 of the sequence encoded video 386 (0) and the distortion level 336 of the encoded video sequence 386 (0).
[0080] The lattice iterator 360 then calculates, for each envelope firing coding point 320 within the firing coding sequence 382 (0), the rate of change of distortion in relation to the bit rate 332 between the wrap triggering coding 320 and the neighbor above wrapping trigger coding point 320. For example, the lattice iterator 360 could calculate the rate of change of distortion in relation to the bit rate 332 between nodes 00 and 01, 10 and 11, 20 and 21, 30 and 31, and 40 and 41 Notably, the calculated rate of change for the envelope firing coding point 320 which includes a particular coded firing sequence 326 represents that derived from the associated distortion curve 450 to that firing sequence 132, taken at envelope firing encoding point 320.
[0081] The lattice iterator 360 selects the derivative having the greatest magnitude, and then selects the neighbor above associated with that derivative for inclusion in a subsequent trigger coding sequence 382. For example, in Figure 5B, the lattice iterator 360 determines
Petition 870190082340, of 08/23/2019, p. 64/125
38/68 that the derivative associated with the envelope firing coding point 320 30 is greater and therefore includes the envelope firing coding point 320 31 (the neighbor above the envelope firing coding point 320 30) in the trigger coding sequence 382 (1). In particular, as shown, the lattice iterator 360 generates the trigger coding sequence 382 (1) which includes the envelope triggering coding points 320 00, 10, 20, 31 and 40.
[0082] The lattice iterator 360 then generates the encoded video sequence 386 (1) based on the trigger encoding sequence 382 (1). More precisely, the lattice iterator 360 aggregates the encoded shooting sequences 326 sequentially included in the envelope firing encoding points 00, 10, 20, 31 and 40 to generate the encoded video sequence 386 (1). Subsequently, the lattice iterator 360 calculates the bit rate 332 and the level of distortion 336 of the encoded video sequence 386 (1). The lattice iterator 360 then generates the video encoding point 380 (1) which includes, without limitation, the trigger encoding sequence 382 (1), the encoded video sequence 386 (1), the bit rate 332 of the encoded video sequence 386 (1) and the distortion level 336 of encoded video sequence 386 (1).
[0083] The truss iterator 360 performs this iterative technique, thus ascending the 370 sequence truss, as shown in Figures 5C-5D.
[0084] In Figure 5C, the lattice iterator 360 determines that the derivative associated with the wrapping trigger coding point 320 00 is greater compared to
Petition 870190082340, of 08/23/2019, p. 65/125
39/68 other derivatives and, therefore, selects the envelope firing encoding point 320 01 for inclusion in the firing encoding sequence 382 (2). As shown,
the iterator trellis 360 generates the sequence of coding in shooting 382 (2) that includes the points of coding in firing wrap 320 01, 10, 20, 31 and 40.[0085] 0 iterator of 360 truss then generates The
encoded video sequence 386 (2) based on the trigger encoding sequence 382 (2). More precisely, the lattice iterator 360 aggregates the encoded trigger sequences 326 included, sequentially, at the envelope trigger encoding points 01, 10, 20, 31 and 40 to generate the encoded video sequence 386 (2). Subsequently, the lattice iterator 360 calculates the bit rate 332 and the level of distortion 336 of the encoded video sequence 386 (2). The lattice iterator 360 then generates the video encoding point 380 (2) which includes, without limitation, the shooting encoding sequence 382 (2), the encoded video sequence 386 (2), the bit rate 332 of the encoded video sequence 386 (2) and the distortion level 336 of the encoded video sequence 386 (2).
[0086] The lattice iterator 360 continues this process until, as shown in Figure 5D, it generates the video coding point 380 (T). Video encoding point 380 (T) includes, without limitation, trigger encoding sequence 382 (T), encoded video sequence 386 (T), bit rate 332 of encoded video sequence 386 (T) and the distortion level 336 of the 386 (T) encoded video sequence.
[0087] In this way, the 360 truss iterator
Petition 870190082340, of 08/23/2019, p. 66/125
40/68 incrementally improves the trigger encoding sequence 382 by selecting a single envelope trigger encoding point 320 for which the bit rate is increased and the distortion is decreased, thus generating a collection of rate encoded 386 video sequences increasing bit rate and decreasing distortion.
[0088] In one embodiment, the lattice iterator 360 adds wrapping trigger coding points 320 before ascending the sequence lattice 370 in order to create a termination condition. In doing so, the lattice iterator 360 can duplicate wrapping trigger encoding points 320 having the highest bit rate 332 to cause the rate of change between the second to the last and last triggering encoding points of wrap 320 is zero. When this rate of change of zero is detected for all trigger sequences 132, that is, when the maximum magnitude of the rate of change is exactly zero, the lattice iterator 360 identifies the termination condition and for the iteration.
GENERATE NEW ENCODING POINTS [0089] Figure 6 illustrates the global convex wrapper 390 that is generated by dynamic optimizer 150 in Figure 1 based on the different video coding points 380 shown in Figures 5A-5D, according to various modalities of present invention. As shown, a graphic 600 includes the bit rate axis 410 and the distortion axis 420.
[0090] As described in detail in conjunction with Figures 5A-D, the lattice iterator 360 generates the encoded trigger sequence 382 in an upward manner to reduce the level of distortion 336 and increase the bit rate
Petition 870190082340, of 08/23/2019, p. 67/125
41/68
332. Consequently, the associated encoded video streams 386 cover a range from high distortion level 336 and low bit rate 332 to low distortion level 336 and high bit rate 332. Among other things, each of the Video encoding 380 (x) includes the bit rate 332 of the encoded video sequence 386 (x) and the distortion level 336 of the encoded video sequence 386 (x).
[0091] As shown, the dynamic optimizer 150 plots the different video encoding points 380 against the bit rate axis 410 and the distortion axis 420, to generate the global convex envelope 390. The dynamic optimizer 150 connects the points ( i.e., the video encoding points 380) included in the global convex wrapper 390 to generate a curve 630. Consequently, curve 630 represents the level of distortion 336 as a function of bit rate 332 across all encoded video streams 386.
[0092] In general, based on curve 630, iteration controller 140 can select, for a given bit rate 332, the video encoding point 380 which includes the encoded video sequence 386 which minimizes the level of distortion 336 Conversely, iteration controller 140 may select, for a given distortion level 336, the video encoding point 380 that includes the encoded video sequence that minimizes the bit rate 332 to the indicated distortion level 336.
[0093] In operation, iteration controller 140 selects the optimized video encoding point 380 based on the target metric value 170. The target metric value 170 can be a target bit rate 332 or a level of
Petition 870190082340, of 08/23/2019, p. 68/125
42/68 target distortion 336. In the embodiment shown in Figure 6, the target metric value 170 is a target bit rate 332. Consequently, iteration controller 140 selects the optimized video encoding point 380 that includes the video sequence encoded 386 having bit rate 332 that is closest to target metric value 170.
[0094] As shown, based on target metric value 170, iteration controller 140 selects the optimized video encoding point 380 (2) which includes the encoded video sequence 38 6 (2). Notably, the encoded video sequence 386 (2) is the encoded video sequence 386 that minimizes the level of distortion 336 for the target bit rate 332. In alternative embodiments, the iteration controller 140 selects the optimized video encoding point 380 which includes the encoded video sequence 386 having the level of distortion 336 which is closest to the target metric value 170. Namely, the optimized video encoding point 380 also includes the optimized shooting encoding sequence 282.
[0095] As previously described in conjunction with Figures 2 and 5A-5D, the shooting coding sequence 282 (2) specifies the shooting coding points 320 which include coded shooting sequences 326 which are included in the coded video sequence 38 6 (2). As shown, the encoded video sequence 386 (2) includes the encoded shooting sequence 326 derived from a 640x360 version of the firing sequence 122 (0) and encoded at the QP value of 26, followed by the encoded shooting sequence 326 derived from a 640x360 version of the firing sequence 122 (1) and encoded in the QP value of
Petition 870190082340, of 08/23/2019, p. 69/125
43/68
51, followed by the encoded firing sequence 326 derived from a 640x360 version of firing sequence 122 (2) and encoded in the QP value of 51, etc.
[0096] In general, each of the optimized trigger coding points 320 included in the optimized trigger coding sequence 282 is associated with a different trigger sequence 132. For each trigger sequence 132 (x), iteration controller 140 assesses the location of the associated optimized trigger coding point 320 along the associated convex envelope 350 (x) to identify one or more nearby trigger coding points 320 in the convex envelope 350 (x). Based on resolutions 322 and QP values 324 from the
coding in firing optimized 320 and The dots in coding in firing Upcoming 320, the controller in iteration 140 Can generate any number of new points in coding 220 to the sequence firing time 132 (x). 0 controller in iteration 140 includes Those new points in
coding 220 in coding list 210 (x). In this way, the iteration controller 140 iteratively refines the range of encoding points 220 in order to efficiently converge to the optimized encoded video sequence 180 that best corresponds to the target metric value 170.
[0097] Figure 7 illustrates how the iteration controller 150 of Figure 1 generates additional coding points 220 for the firing sequence 132 (0), according to various embodiments of the present invention. As described in detail in conjunction with Figure 6, iteration controller 150 identifies the trigger coding point
Petition 870190082340, of 08/23/2019, p. 70/125
44/68
320 (1) including the 640x360 122 (0) firing sequence
coded at the QP value 324 in 2 6 how O Score in coding in optimized shooting 320 to the sequence in shooting 132 (0) •[0098]Subsequently, the controller in iteration
140 generates zero to four new coding points 220 and adds the new coding points 220 to coding list 210 (0) which specifies coding points 220 for the firing sequence 132 (0). Iteration controller 140 implements the following algorithm to select new coding points 220. First, iteration controller 140 identifies the left trigger coding point 320 which is located on the left and adjacent to the optimized trigger coding point 320 along of the convex wrapper 350. If the left trigger coding point 320 has the same resolution 322 as the optimized trigger coding point 320, then iteration controller 140 generates a new coding point 210 that has resolution 322. The controller iteration 140 defines the QP value 324 of the new coding point equal to the average of the QP values 324 of the optimized trigger coding point 320 and the left trigger coding point 320.
[0099] If, however, the left trigger coding point 320 does not have the same resolution 322 as the optimized trigger coding point 320, then iteration controller 140 generates two new coding points 210. The iteration controller 140 defines resolution 322 of the first new coding point 210 equal to resolution 322 of the optimized trigger coding point
Petition 870190082340, of 08/23/2019, p. 71/125
45/68
320. The iteration controller 140 sets the QP value 324 of the first new coding point 210 to the average of the QP value 324 of the optimized trigger coding point 320 and the maximum QP value allowed by the video codex. The iteration controller 140 sets the resolution 322 of the second new coding point 210 equal to the resolution which is immediately below the resolution 322 of the optimized trigger coding point 320. The iteration controller 142 defines the QP value 324 of the second new point code 210 for the minimum QP value allowed by the video codex.
[00100] The iteration controller 140 then identifies the right trigger coding point 320 which is located on the right and adjacent to the optimized trigger coding point 320 along the convex envelope 350. If the right trigger coding point 320 has the same resolution 322 as the optimized trigger coding point 320, then iteration controller 140 generates a new coding point 210 that has resolution 322. Iteration controller 140 sets the QP value 324 of the new coding point equal to the average of the QP values 324 of the optimized trigger coding point 320 and the right trigger coding point 320.
[00101] If, however, the right trigger coding point 320 does not have the same resolution 322 as the optimized trigger coding point 320, then iteration controller 140 generates two new coding points 210. The iteration controller 140 defines resolution 322 of the first new coding point 210 equal to resolution 322 of the optimized trigger coding point
Petition 870190082340, of 08/23/2019, p. 72/125
46/68
320. The iteration controller 140 sets the QP value of the first new coding point 210 to the average of the QP value 324 of the optimized trigger coding point 320 and the minimum QP value allowed by the video codec. The iteration controller 140 sets the resolution 322 of the second new coding point 210 equal to the resolution which is immediately higher than the resolution 322 of the optimized trigger coding point 320. The iteration controller 140 sets the QP value 324 of the second new point code 210 for the maximum QP value allowed by the video codex.
[00102] Coding point operations 710 represent exemplary operations performed by iteration controller 140. As shown, the optimized trigger coding point 320 (1) has a resolution of 322 (1) of 640x360 and the value of QP 324 (1 ) of 26. The left trigger coding point 320 (0) has a resolution 322 (0) of 640x360 and the QP value 324 (0) of 51. Therefore, iteration controller 140 adds the new coding points 220 (6), 220 (7) and 220 (8) for coding list 210 (0) (there are already six entries in coding list 210 (0)). Coding point 220 (6) has a resolution of 322 (6) of 640x360 and the QP value 324 (6) of (51 + 26) / 2 = 39. Coding point 220 (7) has a resolution of 322 (7) of 640x360 and the QP value of 324 (7) of (26 + l) / 2 = 14. The coding point 220 (8) has the 322 (8) resolution of 960x540 and the value of QP 324 ( 8) of (26 + 51) / 2 = 39.
[00103] In alternative modalities, iteration controller 140 can implement any number and type of algorithms to select new coding points 220.
Petition 870190082340, of 08/23/2019, p. 73/125
47/68
For example, in some embodiments, iteration controller 140 does not restrict the search for additional coding points 220 based on the convex wrapper 250. Instead, iteration controller 140 expands the search to the left of the trigger coding point optimized 320, among all available resolutions, to identify the trigger coding point 320 having a resolution 322 that is closer to, but lower than, the resolution 322 of the optimized trigger coding point 320 and a bit rate 324 below. The iteration controller 140 then generates a new coding point 220 having the resolution 322 of the identified trigger coding point 320 and the QP value 324 which is the average of the QP value 324 of the selected trigger coding point 320 and a at the same resolution 322 and a slightly lower QP value 324.
[00104] In a complementary way, iteration controller 140 expands the search to the right of the optimized trigger coding point 320, among all available resolutions, to identify the trigger coding point 320 having a resolution 322 which is more close to, but higher than, the resolution 322 of the optimized trigger coding point 320 and a bit rate 324 just higher than the bit rate 324 of the optimized trigger coding point 320. Iteration controller 140 then generates a new coding 220 having the resolution 322 of the identified shooting coding point 320 and QP value which is the average between the QP value 324 of the selected shooting coding point 320 and one at the same resolution 322 and a slightly higher QP value 324 .
[00105] For each of the shooting sequences 132
Petition 870190082340, of 08/23/2019, p. 74/125
48/68 (x), after identifying any new coding points 220 for trigger sequence 132 (x), iteration controller 140 discards any of the new coding points 220 that are already included in coding list 210 (x ) associated with the firing sequence 132 (x). The iteration controller 140 then adds any remaining new coding points 220 (x) associated with the firing sequence 132 (x) to the coding list 210 (x). If iteration controller 140 adds any new coding points 220 to any of the coding lists 210, then iteration controller 140 configures dynamic optimizer 150 to re-run based on the updated coding lists 210. If, however, the controller iteration 140 does not add new encoding points 220 to any of the coding lists 210, so iteration controller 140 determines that iteration controller 140 converged to the optimized video encoding point 380 having the encoded video sequence 386 which best matches the target metric value 170. Consequently, iteration controller 140 sets the optimized video sequence 180 equal to the encoded video sequence 386 included in the optimized video encoding point 380.
[00106] In alternative modalities, iteration controller 140 may determine that iteration controller 140 has converged to the optimized video encoding point 380 having the encoded video sequence 386 that sufficiently matches the target metric value 170 of any technically feasible way. For example, in some embodiments, iteration controller 140 may stop
Petition 870190082340, of 08/23/2019, p. 75/125
49/68 the iteration (for example, rerun the dynamic optimizer 150) when a total number of iterations equals a maximum iteration configuration parameter (for example, 5 iterations). In other embodiments, iteration controller 140 can measure the improvement achieved in each iteration and stop iterating after the improvement is less than an improvement configuration parameter (for example, 1%). In still other embodiments, iteration controller 140 can track the amount of processor resources consumed by the iterative encoding application 120 and stop the iteration after the amount of processor resources is greater than a processor threshold configuration parameter.
[00107] After determining the optimized video sequence 180, iteration controller 140 transmits the optimized video sequence 180 to CDN 190 in any technically feasible manner. In alternative embodiments, iteration controller 140 can periodically transmit the encoded video sequence 386 included in the optimized video encoding point 380 to CDN 190. In this way, iteration controller 140 allows the terminal devices to display the title of media while iteration controller 140 continues to iterate.
[00108] Figure 8 is a more detailed illustration of the optimized encoded video sequence 180 of Figure 1, according to various embodiments of the present invention. Advantageously, the iterative encoding application 120 optimizes each of the encoded shooting sequences 326 included in the optimized encoded video sequence 180 based on the target metric value 170.
Petition 870190082340, of 08/23/2019, p. 76/125
50/68 [00109] As shown, the optimized encoded video sequence 180 includes encoded shooting sequence 326 derived from a 960x540 version of shooting sequence 122 (0) and encoded at a QP value of 40, followed by the shooting sequence encoded 326 derived from a 640x360 version of firing sequence 122 (1) and encoded at a QP value of 11, followed by encoded firing sequence 326 derived from a 1280x720 version of firing sequence 122 (2) and encoded in value QP of 47, etc.
[00110] Figures 9A-9B establish a flowchart of method steps for encoding a source video sequence, according to various modalities of the present invention. Although the method steps are described with reference to the systems of Figures 1-8, those skilled in the art will understand that any system configured to implement the method steps, in any order, falls within the scope of the present invention.
[00111] As shown, a method 900 starts at step 902, where trigger analyzer 130 partitions source video sequence 122 into trigger sequences 132. In step 904, for each of the trigger sequences 132, the trigger controller iteration 140 generates coding lists 210. Each of coding points 220 included in coding lists 210 includes resolution 322 and the value of QP 324. Alternatively, each coding point 220 can specify any number and type of encoding parameters instead of or in addition to the QP 324 value.
[00112] In step 906, for each of the lists of
Petition 870190082340, of 08/23/2019, p. 77/125
51/68 coding 210, dynamic optimizer 150 identifies the new coding points 220 and generates the corresponding encoded shooting sequences 326. In step 908, for each of the new encoded shooting sequences 326, dynamic optimizer 150 calculates the rate of bits 332, quality score 334 and distortion level 336. Subsequently, the dynamic optimizer 150 generates the new trigger coding point 320 which includes, without limitation, the coded trigger sequence 326, resolution 322, the value of QP 324, bit rate 332, quality score 334, distortion level 336.
[00113] In step 910, for each permutation of the trigger sequence 132 and the resolution 322, the convex envelope generator 340 generates the associated distortion curve 450 based on the corresponding trigger coding points 320. In step 912, for each one of the firing sequences 132 (x), the convex envelope generator 340 generates the convex envelope 350 (x) based on the associated distortion curves 450. In step 914, the truss iterator 360 generates the sequence truss 370 based on the convex wraps 350. In step 916, the truss iterator 360 iteratively ascends the sequence truss 370 to generate the encoded video streams 386 and the associated video encoding points 380. In step 918, the truss iterator 360 generates the wrap global convex 290 based on video encoding points 380.
[00114] In step 920, iteration controller 140 selects the optimized video encoding point 380 based on the global convex wrapper 290 and the target metric value 170. In step 922, for each of the sequences of
Petition 870190082340, of 08/23/2019, p. 78/125
52/68 shot 132, iteration controller 140 selects the optimized shot coding point 320 based on the shot coding sequence 382 included in the ideal video coding point 380. In step 924, for each of the shooting sequences 232 (x), iteration controller 140 generates a new coding point 220 based on the convex envelope 350 (x) and the optimized trigger coding point 320 associated with the firing sequence 232 (x). In step 926, iteration controller 140 determines whether there are any new coding points 220. If, in step 926, iteration controller 140 determines that there are new coding points 220, then method 900 returns to step 960, where the dynamic optimizer 150 generates new encoded shooting sequences 326 for the new encoding point (s) 220.
[00115] If, however, in step 926, iteration controller 140 determines that there is no new coding point (s) 220, then method 900 proceeds to step 928. In step 928, iteration controller 140 configures the optimized encoded video sequence 180 equal to the encoded video sequence 386 included in the optimized video encoding point 380. In step 930, iteration controller 140 transmits the optimized encoded video sequence 180 to the content delivery network 190 for streaming to terminal devices. Method 900 then ends.
[00116] In short, the techniques disclosed allow for efficient and ideal encoding of source video sequences based on a target metric value. An iterative coding application includes, without limitation, a
Petition 870190082340, of 08/23/2019, p. 79/125
53/68 trigger analyzer, an iteration controller and a dynamic optimizer. First, the trigger analyzer divides a source video sequence into several trigger sequences. Subsequently, for each trigger sequence, the iteration controller initializes a coding list to include coding points with relatively dispersed distribution. Each coding point specifies a resolution and a quantification parameter (QP). The iteration controller then sets up the dynamic optimizer to generate a global convex envelope based on the coding lists.
[00117] First, the dynamic optimizer generates new trigger coding points based on the new coding points included in the coding lists. A new coding point is a coding point for which the dynamic optimizer has not previously generated an associated coded firing sequence. Each trigger encoding point includes, without limitation, an encoded trigger sequence, resolution, QP value, quality score, distortion level and bit rate. Subsequently, for each trigger sequence, the dynamic optimizer generates a convex envelope that includes a subset of the trigger coding points associated with the trigger sequence. In general, for a given trigger sequence, the trigger encoding points included in the associated convex envelope maximize the bit rate for different levels of distortion.
[00118] The dynamic optimizer then evaluates the convex wraps across all firing sequences to determine
Petition 870190082340, of 08/23/2019, p. 80/125
54/68 shooting. Each trigger coding sequence specifies the trigger coding points for the different trigger sequences. For each trigger encoding sequence, the dynamic optimizer aggregates the different encoded trigger sequences included in the trigger encoding points to generate an encoded video sequence. For each trigger encoding sequence, the dynamic optimizer then generates a video encoding point that includes the trigger encoding sequence, the associated encoded video sequence, an overall bit rate for the encoded trigger sequence, and a level of global distortion for the encoded firing sequence. Subsequently, the dynamic optimizer generates a global convex envelope based on the video encoding points. In general, for the source video stream, each of the video encoding points included in the overall convex wrap minimizes the overall bit rate to a different level of overall distortion.
[00119] The iteration controller selects an optimized video encoding point from the global convex envelope based on a target metric value (for example, bit rate or quality score). For each trigger sequence, the optimized video encoding point specifies an optimized trigger encoding point. For each trigger sequence, the iteration controller evaluates the location of the optimized trigger coding point along the associated convex envelope to identify one or more nearby trigger codes. Based on the resolutions and QP values of the optimized trigger coding point and the
Petition 870190082340, of 08/23/2019, p. 81/125
55/68 trigger coding nearby, the iteration controller can generate any number of new coding points for the trigger sequence. If the iteration controller identifies any new coding points, the iteration controller configures the dynamic optimizer to generate a new global convex envelope based on the expanded set of coding points.
[00120] If, however, the iteration controller does not identify any new encoding points, then the iteration controller selects the encoded video sequence included in the optimized video encoding point as the optimized encoded video sequence. Finally, the iteration controller transmits the optimized encoded video stream to a content delivery network for distribution to terminal devices.
[00121] At least one technological improvement of the techniques disclosed in relation to the prior art is that, iteratively converging to an optimized coded video sequence that includes individually coded shooting sequences reduces the coding inefficiencies typically associated with conventional coding techniques. Such coding inefficiencies not only waste computer and storage resources unnecessarily, these types of inefficiencies also unnecessarily increase the bandwidth needed to transmit coded versions of media titles to terminal devices.
1. In some embodiments, a computer-implemented method comprises generating a first set of trigger encoding points based on a first set
Petition 870190082340, of 08/23/2019, p. 82/125
56/68 encoding points and a first trigger sequence included in a source video sequence that is associated with a media title, where each trigger encoding point is associated with a different encoded trigger sequence; performing one or more convex wrap operations through the first set of trigger coding points to generate a first convex wrap that is associated with the first trigger sequence; generating a plurality of encoded video sequences based on the first convex envelope and a second convex envelope that is associated with a second trigger sequence included in the source video sequence; calculating a first encoding point that is not included in the first set of encoding points based on the plurality of encoded video streams and a target value for a first video metric; and generating an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently transmitted continuously to a terminal device during playback of the media title.
2. Method implemented by computer in accordance with clause 1, in which the generation of the optimized encoded video sequence comprises generating a first encoded shooting sequence based on the first shooting sequence and the first encoding point; generating a second coded trigger sequence based on the second trigger sequence and a second coding point that is not the same as the first coding point; and aggregate the first encoded firing sequence and the second firing sequence
Petition 870190082340, of 08/23/2019, p. 83/125
57/68 coded.
3. The computer implemented method of clauses 1 or 2, in which the performance of one or more convex wrap operations comprises determining a region that includes the first set of trigger coding points; identify a region boundary, where no trigger coding points included in the first set of trigger coding points are located on a first side of the boundary; and discard any trigger coding points included in the first set of trigger coding points that are not located along the boundary to generate the first convex envelope.
4. The computer-implemented method of any one of clauses 1-3, wherein the generation of a plurality of encoded video sequences comprises calculating a first slope value between a first trigger coding point and a second trigger coding point trigger, in which both the first trigger coding point and the second trigger coding point are included in the first convex envelope; selecting a first encoded trigger sequence associated with the second trigger encoding point based on the first slope value; and aggregating the first encoded trigger sequence with a second encoded trigger sequence to generate a first encoded video sequence, wherein the second encoded trigger sequence is associated with a third trigger encoding point included in the second convex envelope.
5. The method implemented by anyone's computer
Petition 870190082340, of 08/23/2019, p. 84/125
58/68 of clauses 1-4, where the calculation of the first encoding point comprises generating a global convex envelope based on the plurality of encoded video sequences, wherein the global convex envelope includes a first set of video encoding points ; selecting a first video encoding point included in the first set of video encoding points based on the target value; and determining the first coding point based on the first video coding point and the first convex envelope.
6. The computer implemented method of any one of clauses 1-5, wherein the calculation of the first encoding point comprises identifying a first encoded video sequence included in the plurality of encoded video sequences based on the target value; identifying a first trigger encoding point that is associated with the first encoded video sequence and is included in the first convex envelope; identifying a second trigger coding point included in the first convex envelope based on the distance between the first trigger coding point and the second trigger coding point; and performing at least one arithmetic operation based on the first trigger coding point and the second trigger coding point.
7. The computer-implemented method of any of clauses 1-6, in which the performance of at least one arithmetic operation comprises calculating an average of a first value of a coding parameter and a second value of the coding parameter, and in that the first value of the encoding parameter is associated with the first
Petition 870190082340, of 08/23/2019, p. 85/125
59/68 trigger coding and the second encoding parameter value is associated with the second trigger coding point.
8. The computer implemented method of any one of clauses 1-7, wherein the calculation of the first encoding point comprises identifying a first encoded video sequence included in the plurality of encoded video sequences based on the target value; identifying a first trigger encoding point that is associated with the first encoded video sequence and is included in the first convex envelope; identifying a second trigger coding point included in the first convex envelope based on the distance between the first trigger coding point and the second trigger coding point; determining that a first resolution associated with the first trigger coding point is not the same as a second resolution associated with the second trigger coding point; define, based on the first resolution and the second resolution, an end point value of a coding parameter equal to a maximum value of the coding parameter or a minimum value of the coding parameter; and performing at least one arithmetic operation between a first encoding parameter value that is associated with the first trigger encoding point and the encoding parameter end point value.
9. The computer-implemented method of any of clauses 1-8, in which the first video metric comprises a bit rate, a peak signal-to-noise ratio (PSNR), a multi-method evaluation fusion metric linear video (VMAF), a harmonic VMAF metric (VMAFh)
Petition 870190082340, of 08/23/2019, p. 86/125
60/68 or a distortion metric.
10. In some embodiments, a computer-readable storage medium includes instructions that, when executed by a processor, make the processor perform the steps of generating a first set of trigger coding points based on a first set of coding points and a first shooting sequence included in a source video sequence that is associated with a media title, where each shooting encoding point is associated with a different encoded shooting sequence; performing one or more convex wrap operations through the first set of trigger coding points to generate a first convex wrap which is associated with the first trigger sequence; generating a global convex envelope based on the first convex envelope and a second convex envelope that is associated with a second firing sequence included in the source video sequence; calculate a first coding point that is not included in the first set of coding points based on a target value for a first video metric, the global convex envelope and the first convex envelope; and generating an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently transmitted continuously to a terminal device during playback of the media title.
11. Computer-readable storage medium according to clause 10, in which the generation of the optimized encoded video sequence comprises generating a first
Petition 870190082340, of 08/23/2019, p. 87/125
61/68 firing sequence coded based on the first firing sequence and the first coding point; generating a second coded trigger sequence based on the second trigger sequence and a second coding point that is not the same as the first coding point; and aggregating the first encoded firing sequence and the second encoded firing sequence.
12. The computer readable storage medium of clauses 10 or 11, wherein performing one or more convex wrap operations comprises determining a region that includes the first set of trigger coding points; identify a region boundary, where no trigger coding points included in the first set of trigger coding points are located on a first side of the boundary; and discard any trigger coding points included in the first set of trigger coding points that are not located along the boundary to generate the first convex envelope.
13. The computer-readable storage medium of any one of clauses 10-12, in which each trigger encoding point included in the first convex envelope is associated with both a bit rate and a value for either a distortion metric or a visual quality metric.
14. The computer-readable storage medium of any of clauses 10-13, in which the generation of the global convex envelope comprises calculating a first slope value between a first trigger coding point and a second trigger coding point, Where
Petition 870190082340, of 08/23/2019, p. 88/125
62/68 both the first trigger coding point and the second trigger coding point are included in the first convex wrap; selecting a first encoded trigger sequence associated with the second trigger encoding point based on the first slope value; aggregating the first encoded shooting sequence with a second encoded shooting sequence to generate a first encoded video sequence, wherein the second encoded shooting sequence is associated with a third shooting encoding point included in the second convex envelope; generating a first video encoding point based on the first encoded video sequence; and add the first video encoding point to a partial global convex wrap to generate the convex wrap
global. in 15. 0 middle of storage readable per computer any one of clauses 10- 14, in what the calculation of first point of coding comprises select one
first video encoding point included in the global convex envelope based on the target value; and selecting a first trigger encoding point included in the first convex envelope based on the first video encoding point; identifying a second trigger coding point included in the first convex envelope based on the distance between the first trigger coding point and the second trigger coding point; and performing at least one arithmetic operation based on the first trigger coding point and the second trigger coding point.
16. The computer-readable storage medium of
Petition 870190082340, of 08/23/2019, p. 89/125
63/68 any one of clauses 10-15, wherein performing at least one arithmetic operation comprises calculating an average of a first resolution associated with the first trigger coding point and a second resolution associated with the second trigger coding point shooting.
17. The computer-readable storage medium of any of clauses 10-16, in which the calculation of the first encoding point comprises selecting a first video encoding point included in the global convex envelope based on the target value; and selecting a first trigger encoding point included in the first convex envelope based on the first video encoding point; identifying a second trigger coding point included in the first convex envelope based on the distance between the first trigger coding point and the second trigger coding point; determining that a first resolution associated with the first trigger coding point is not the same as a second resolution associated with the second trigger coding point; define, based on the first resolution and the second resolution, an end point value of a coding parameter equal to a maximum value of the coding parameter or a minimum value of the coding parameter; and performing at least one arithmetic operation between a first encoding parameter value that is associated with the first trigger encoding point and the encoding parameter end point value.
18. The computer-readable storage medium of any of clauses 10-17, where the first coding point specifies at least one of a resolution and one
Petition 870190082340, of 08/23/2019, p. 90/125
64/68 encoding parameter.
19. In some embodiments, a system comprises a memory storing instructions; and a processor that is coupled to the memory and, when executing the instructions, is configured to partition a video sequence that is associated with a media title in a plurality of framesets; generating a first set of trigger coding points based on a first set of coding points and a first set of frames included in the plurality of sets of frames, where each trigger coding point is associated with a set of coded frames different; performing one or more convex wrap operations through the first set of trigger coding points to generate a first convex wrap that is associated with the first set of frames; generating a plurality of encoded video sequences based on the first convex wrap and a second convex wrap which is associated with a second set of frames included in the plurality of sets of frames; calculating a first encoding point that is not included in the first set of encoding points based on the plurality of encoded video streams and a target value for a first video metric; and generating an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently transmitted continuously to a terminal device during playback of the media title.
20. The clause 19 system, where the processor is
Petition 870190082340, of 08/23/2019, p. 91/125
65/68 configured to determine the generation of the optimized encoded video sequence by generating a first set of encoded frames based on the first set of frames and the first encoding point; generating a second set of encoded frames based on the second set of frames and a second encoding point that is not the same as the first encoding point; and aggregate the first set of encoded frames and the second set of encoded frames.
[00122] Any and all combinations of any of the claimed elements mentioned in any of the claims and / or any elements described in this application, in any way, are within the contemplated scope of the present invention and protection.
[00123] The descriptions of the various modalities have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed modalities. Many modifications and variations will be evident to those skilled in the art without departing from the scope and spirit of the modalities described.
[00124] Aspects of the present modalities can be incorporated as a computer program system, method or product. Therefore, aspects of the present disclosure may take the form of an entirely hardware modality, an entirely software modality (including firmware, resident software, microcode, etc.) or a modality combining aspects of software and hardware that can generally be referred to here as a module or system. In addition, aspects of this disclosure may take the form of a
Petition 870190082340, of 08/23/2019, p. 92/125
66/68 computer program embedded in one or more computer-readable media having computer-readable program code embedded in them.
[00125] Any combination of one or more computer-readable media can be used. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable floppy disk, a hard disk, a random access memory (RAM), a memory read-only (ROM), a programmable and erasable read-only memory (EPROM or flash memory), an optical fiber, a read-only portable compact disc (CD-ROM) memory, an optical storage device, a storage device magnetic storage or any suitable combination of the above. In the context of this document, a computer-readable storage medium may be any tangible medium that may contain or store a program for use by or in connection with a system, apparatus or device for carrying out instructions.
[00126] Aspects of the present disclosure are described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and
Petition 870190082340, of 08/23/2019, p. 93/125
67/68 computer program products according to disclosure modalities. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a general purpose computer processor, special purpose computer, or other programmable data processing device to produce a machine. The instructions, when executed through the computer processor or other programmable data processing device, allow the implementation of the functions / actions specified in the flowchart and / or block or block diagram blocks. Such processors may be, without limitation, general purpose processors, special purpose processors, application specific processors or programmable field gate arrays.
[00127] The flowchart and block diagrams in the figures illustrate the architecture, functionality and operation of possible implementations of computer program systems, methods and products in accordance with the various modalities of the present disclosure. In this regard, each block in the flowchart or block diagrams can represent a module, segment or piece of code, which comprises one or more executable instructions to implement the specified logical function (s). It should also be noted that, in some alternative implementations, the functions observed in the block may occur outside the order indicated in the figures. For example, two blocks shown in
Petition 870190082340, of 08/23/2019, p. 94/125
68/68 succession can, in fact, be executed substantially at the same time, or the blocks can sometimes be executed in reverse order, depending on the functionality involved. It will also be noted that each block of block diagram and / or flow chart illustration, and combinations of blocks in the block diagram and / or flow chart illustration, can be implemented by special purpose hardware based systems that perform the functions or acts specified, or combinations of special-purpose hardware and computer instructions.
[00128] While the previous one is directed to modalities of the present description, others and more modalities of the disclosure can be conceived without departing from its basic scope, and its scope is determined by the following claims.

权利要求:
Claims (11)
[1]
AMENDED CLAIMS
1. Computer implemented method, characterized by the fact that it comprises:
generate a first set of shooting encoding points based on a first set of encoding points and a first shooting sequence included in a source video sequence that is associated with a media title, where each encoding point of triggering is associated with a different coded triggering sequence;
performing one or more convex wrap operations through the first set of trigger coding points to generate a first convex wrap which is associated with the first trigger sequence;
generating a plurality of encoded video sequences based on the first convex envelope and a second convex envelope that is associated with a second trigger sequence included in the source video sequence;
calculating a first encoding point that is not included in the first set of encoding points based on the plurality of encoded video streams and a target value for a first video metric; and generating an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently transmitted continuously to a terminal device during playback of the media title.
[2]
2. Method implemented by computer, according to claim 1, characterized by the fact that the generation of the optimized encoded video sequence comprises:
Petition 870190082340, of 08/23/2019, p. 119/125
2/7 generate a first coded firing sequence based on the first firing sequence and the first coding point;
generating a second coded trigger sequence based on the second trigger sequence and a second coding point that is not the same as the first coding point; and aggregating the first encoded firing sequence and the second encoded firing sequence.
[3]
3. Computer implemented method, according to claim 1, characterized by the fact that the performance of one or more convex wrap operations comprises:
determining a region that includes the first set of trigger coding points;
identify a region boundary, where no trigger coding points included in the first set of trigger coding points are located on a first side of the boundary; and discard any trigger coding points included in the first set of trigger coding points that are not located along the boundary to generate the first convex envelope.
[4]
4. Method implemented by computer, according to claim 1, characterized by the fact that the generation of the plurality of encoded video sequences comprises:
calculating a first slope value between a first trigger coding point and a second trigger coding point, wherein both the first trigger coding point and the second trigger coding point are included in the first convex envelope;
Petition 870190082340, of 08/23/2019, p. 120/125
3/7 select a first encoded trigger sequence associated with the second trigger encoding point based on the first slope value; and aggregating the first encoded trigger sequence to a second encoded trigger sequence to generate a first encoded video sequence, wherein the second encoded trigger sequence is associated with a third trigger encoding point included in the second convex envelope.
[5]
5. Method implemented by computer, according to claim 1, characterized by the fact that the calculation of the first coding point comprises:
generate one wrap convex global based at plurality in strings of video coded, where O wrap global convex includes a first set in
video encoding points;
selecting a first video encoding point included in the first set of video encoding points based on the target value; and determining the first encoding point based on the first video encoding point and the first convex wrap.
[6]
6. Method implemented by computer, according to claim 1, characterized by the fact that the calculation of the first coding point comprises:
identifying a first encoded video sequence included in the plurality of encoded video sequences based on the target value;
identify a first trigger encoding point that is associated with the first encoded video sequence and
Petition 870190082340, of 08/23/2019, p. 121/125
4 / Ί is included in the first convex wrap;
identifying a second trigger coding point included in the first convex envelope based on the distance between the first trigger coding point and the second trigger coding point; and performing at least one arithmetic operation based on the first trigger coding point and the second trigger coding point.
[7]
7. Computer-implemented method according to claim 6, characterized by the fact that performing at least one arithmetic operation comprises calculating an average of a first value of a coding parameter and a second value of the coding parameter, and wherein the first encoding parameter value is associated with the first trigger encoding point and the second encoding parameter value is associated with the second trigger encoding point.
[8]
8. Method implemented by computer, according to claim 1, characterized by the fact that the calculation of the first coding point comprises:
identifying a first encoded video sequence included in the plurality of encoded video sequences based on the target value;
identifying a first trigger encoding point that is associated with the first encoded video sequence and is included in the first convex envelope;
identifying a second trigger coding point included in the first convex envelope based on the distance between the first trigger coding point and the second trigger coding point;
Petition 870190082340, of 08/23/2019, p. 122/125
5/7 determine that a first resolution associated with the first trigger coding point is not the same as a second resolution associated with the second trigger coding point;
define, based on the first resolution and the second resolution, an end point value of a coding parameter equal to a maximum value of the coding parameter or a minimum value of the coding parameter; and performing at least one arithmetic operation between a first encoding parameter value that is associated with the first trigger encoding point and the encoding parameter end point value.
[9]
9. Method implemented by computer, according to claim 1, characterized by the fact that the first video metric comprises a bit rate, a peak signal-to-noise ratio (PSNR), a multi-method evaluation fusion metric linear video (VMAF), a harmonic VMAF metric (VMAFh), or a distortion metric.
[10]
10. Computer-readable storage medium characterized by the fact that it comprises instructions that, when executed by a processor, make the processor perform the following steps:
generate a first set of shooting encoding points based on a first set of encoding points and a first shooting sequence included in a source video sequence that is associated with a media title, where each encoding point of triggering is associated with a different coded triggering sequence;
perform one or more convex wrap operations
Petition 870190082340, of 08/23/2019, p. 123/125
6/7 through the first set of trigger coding points to generate a first convex envelope that is associated with the first trigger sequence;
generating a global convex envelope based on the first convex envelope and a second convex envelope that is associated with a second shooting sequence included in the source video sequence;
calculate a first encoding point that is not included in the first set of encoding points based on a target value for a first video metric, the global convex wrap and the first convex wrap; and generating an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently transmitted continuously to a terminal device during playback of the media title.
[11]
11. System characterized by the fact that it comprises:
a memory storing instructions; and a processor that is attached to the memory and, when executing the instructions, is configured to:
partition a video sequence associated with a media title into a plurality of framesets;
generate a first set of trigger coding points based on a first set of coding points and a first set of frames included in the plurality of sets of frames, where each trigger coding point is associated with a set of
Petition 870190082340, of 08/23/2019, p. 124/125
7/7 different encoded frames;
performing one or more convex wrap operations through the first set of trigger coding points to generate a first convex wrap that is associated with the first set of frames;
generating a plurality of encoded video sequences based on the first convex wrap and a second convex wrap which is associated with a second set of frames included in the plurality of sets of frames;
calculating a first encoding point that is not included in the first set of encoding points based on the plurality of encoded video streams and a target value for a first video metric; and generating an optimized encoded video sequence based on the first encoding point, wherein at least a portion of the optimized encoded video sequence is subsequently transmitted continuously to a terminal device during playback of the media title.

类似技术:

---

公开号 | 公开日 | 专利标题

---

BR112019017608A2|2020-03-24|ITERATIVE TECHNIQUES FOR VIDEO CONTENT ENCODING

---

US10742708B2|2020-08-11|Iterative techniques for generating multiple encoded versions of a media title

---

US11153585B2|2021-10-19|Optimizing encoding operations when generating encoded versions of a media title

---

BR112021013514A2|2021-09-14|OPTIMIZING ENCODING OPERATIONS WHEN GENERATING A TEMPORARY STORAGE RESTRICTED VERSION OF A MEDIA TITLE

---

US20200169593A1|2020-05-28|Techniques for encoding a media title while constraining bitrate variations

---

BR112021008485A2|2021-08-03|techniques for encoding a media title through multiple encoders

---

US11196790B2|2021-12-07|Techniques for encoding a media title while constraining quality variations

---

US11019374B2|2021-05-25|Techniques for efficiently performing subsequence-based encoding for a media title

---

US20220038725A1|2022-02-03|Optimizing encoding operations when generating encoded versions of a media title

---

US20220086453A1|2022-03-17|Techniques for selecting resolutions for encoding different shot sequences

---

BR112021015369A2|2021-09-28|TECHNIQUES FOR EFFICIENTLY PERFORMING SUBSEQUENCE-BASED ENCODING FOR A MEDIA TITLE

同族专利:

---

公开号 | 公开日

---

AU2018223831B2|2020-12-24|

---

EP3586517A1|2020-01-01|

---

CA3052657C|2022-01-18|

---

SG11201907409UA|2019-09-27|

---

CN110313183A|2019-10-08|

---

CN110313183B|2021-11-12|

---

KR102260841B1|2021-06-08|

---

CA3052657A1|2018-08-30|

---

WO2018156997A1|2018-08-30|

---

US20210144386A1|2021-05-13|

---

KR20190120780A|2019-10-24|

---

AU2018223831A1|2019-08-22|

---

US10715814B2|2020-07-14|

---

JP6942807B2|2021-09-29|

---

US20180242015A1|2018-08-23|

---

US10897618B2|2021-01-19|

---

MX2019010116A|2019-10-14|

---

US20180240502A1|2018-08-23|

---

JP2020508605A|2020-03-19|

---

US11184621B2|2021-11-23|

---

US20180241795A1|2018-08-23|

---

WO2018156996A1|2018-08-30|

---

US20180242002A1|2018-08-23|

---

US10917644B2|2021-02-09|

---

US20210160510A1|2021-05-27|

---

AU2021201760A1|2021-04-15|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

US5612742A|1994-10-19|1997-03-18|Imedia Corporation|Method and apparatus for encoding and formatting data representing a video program to provide multiple overlapping presentations of the video program|

---

US6625322B1|1999-06-08|2003-09-23|Matsushita Electric Industrial Co., Ltd.|Image coding apparatus|

---

JP2004511976A|2000-10-10|2004-04-15|コーニンクレッカ　フィリップス　エレクトロニクス　エヌ　ヴィ|Video bit rate control method and apparatus for digital video recording|

---

US7103669B2|2001-02-16|2006-09-05|Hewlett-Packard Development Company, L.P.|Video communication method and system employing multiple state encoding and path diversity|

---

US7400774B2|2002-09-06|2008-07-15|The Regents Of The University Of California|Encoding and decoding of digital data using cues derivable at a decoder|

---

CN1778117A|2003-04-18|2006-05-24|皇家飞利浦电子股份有限公司|System and method for rate-distortion optimized data partitioning for video coding using parametric rate-distortion model|

---

US7242809B2|2003-06-25|2007-07-10|Microsoft Corporation|Digital video segmentation and dynamic segment labeling|

---

WO2005029868A1|2003-09-23|2005-03-31|Koninklijke Philips Electronics, N.V.|Rate-distortion video data partitioning using convex hull search|

---

US7394410B1|2004-02-13|2008-07-01|Samplify Systems, Inc.|Enhanced data converters using compression and decompression|

---

JP4037839B2|2004-03-11|2008-01-23|株式会社東芝|Image coding method and apparatus|

---

CN102833538B|2004-06-27|2015-04-22|苹果公司|Multi-pass video encoding|

---

DE102005042134B4|2005-09-05|2007-08-02|Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.|Apparatus, method and computer program for coding parameter determination for a hybrid coding scheme|

---

WO2007075196A1|2005-09-07|2007-07-05|Vidyo, Inc.|System and method for a high reliability base layer trunk|

---

US7876819B2|2005-09-22|2011-01-25|Qualcomm Incorporated|Two pass rate control techniques for video coding using rate-distortion characteristics|

---

US9113147B2|2005-09-27|2015-08-18|Qualcomm Incorporated|Scalability techniques based on content information|

---

US9036712B1|2006-05-08|2015-05-19|Geo Semiconductor Inc.|Methods and/or apparatus for controlling zero-residual coding in predictive image/video coding|

---

US20080043832A1|2006-08-16|2008-02-21|Microsoft Corporation|Techniques for variable resolution encoding and decoding of digital video|

---

US8300904B2|2007-07-18|2012-10-30|Agency For Science, Technology And Research|Method and device for determining a similarity value between minutiae templates|

---

JP5369893B2|2008-05-30|2013-12-18|株式会社Ｊｖｃケンウッド|Video encoding device, video encoding method, video encoding program, video decoding device, video decoding method, video decoding program, video re-encoding device, video re-encoding method, video re-encoding Encoding program|

---

CN101365125B|2008-09-27|2012-03-21|腾讯科技（深圳）有限公司|Multipath video communication method and system|

---

US8396114B2|2009-01-29|2013-03-12|Microsoft Corporation|Multiple bit rate video encoding using variable bit rate and dynamic resolution for adaptive video streaming|

---

US8451903B2|2009-05-14|2013-05-28|Massachusetts Institute Of Technology|Selecting transforms for compressing visual data|

---

US8509309B2|2009-05-14|2013-08-13|Massachusetts Institute Of Technology|Selecting transforms for compressing visual data|

---

US20120147985A1|2009-08-18|2012-06-14|Pantech Co., Ltd.|Feedbacking channel information in wireless communication system|

---

US20110052087A1|2009-08-27|2011-03-03|Debargha Mukherjee|Method and system for coding images|

---

KR20110032610A|2009-09-23|2011-03-30|삼성전자주식회사|Apparatus and method for scene segmentation|

---

US8171154B2|2009-09-29|2012-05-01|Net Power And Light, Inc.|Method and system for low-latency transfer protocol|

---

US9584700B2|2009-10-26|2017-02-28|Hewlett-Packard Development Company, L.P.|Color separation table optimized for a printing process according to a print attribute by selecting particular Neugebauer primaries and Neugebauer primary area coverages|

---

FR2963189B1|2010-07-20|2013-06-21|Freebox|METHOD FOR ADAPTIVE ENCODING OF A DIGITAL VIDEO STREAM, IN PARTICULAR FOR XDSL LINE BROADCAST.|

---

US8837601B2|2010-12-10|2014-09-16|Netflix, Inc.|Parallel video encoding based on complexity analysis|

---

JP6134650B2|2011-01-28|2017-05-24|アイ アイオー，リミテッド・ライアビリティ・カンパニーＥｙｅ Ｉｏ，Ｌｌｃ|Applicable bit rate control based on scene|

---

CN108769693A|2011-06-10|2018-11-06|茨特里克斯系统公司|Quality perceive it is video optimized in macro-block-level self-adaptive quantization|

---

US8982942B2|2011-06-17|2015-03-17|Microsoft Technology Licensing, Llc|Adaptive codec selection|

---

US9009009B2|2011-06-27|2015-04-14|The Research Foundation For The State University Of New York|Method for predicting optimized crystal structures|

---

US8818171B2|2011-08-30|2014-08-26|Kourosh Soroushian|Systems and methods for encoding alternative streams of video for playback on playback devices having predetermined display aspect ratios and network connection maximum data rates|

---

US9451284B2|2011-10-10|2016-09-20|Qualcomm Incorporated|Efficient signaling of reference picture sets|

---

CN103918009A|2011-11-07|2014-07-09|汤姆逊许可公司|Predictive position encoding|

---

JP5964446B2|2011-11-09|2016-08-03|フラウンホッファー−ゲゼルシャフト ツァ フェルダールング デァ アンゲヴァンテン フォアシュンク エー．ファオ|Inter-layer prediction between different dynamic sample value ranges|

---

WO2013083199A1|2011-12-09|2013-06-13|Telefonaktiebolaget L M Ericsson |Method and apparatus for detecting quality defects in a video bitstream|

---

US10694258B2|2011-12-14|2020-06-23|Netflix, Inc.|Startup times of streaming digital media playback|

---

US9571827B2|2012-06-08|2017-02-14|Apple Inc.|Techniques for adaptive video streaming|

---

US9125073B2|2012-08-03|2015-09-01|Intel Corporation|Quality-aware adaptive streaming over hypertext transfer protocol using quality attributes in manifest file|

---

EP2939420B1|2013-01-15|2018-03-14|Huawei Technologies Co., Ltd.|Using quality information for adaptive streaming of media content|

---

CN104346613B|2013-08-08|2018-06-15|卡西欧计算机株式会社|Image processing apparatus and image processing method|

---

CN103501445B|2013-10-12|2017-02-22|青岛旲天下智能科技有限公司|Gesture-based interaction two-way interactive digital TV box system and implementation method|

---

CN103501446B|2013-10-12|2016-05-18|青岛旲天下智能科技有限公司|Internet television system based on gesture human-computer interaction technology and its implementation|

---

US10019985B2|2013-11-04|2018-07-10|Google Llc|Asynchronous optimization for sequence training of neural networks|

---

JP6271756B2|2013-12-02|2018-01-31|ドルビー・インターナショナル・アーベー|Method of bit rate signaling and bit stream format enabling the method|

---

US9767101B2|2014-06-20|2017-09-19|Google Inc.|Media store with a canonical layer for content|

---

US20160073106A1|2014-09-08|2016-03-10|Apple Inc.|Techniques for adaptive video streaming|

---

CN104185024B|2014-09-16|2017-09-15|福州大学|A kind of HEVC quantization parameter optimization methods based on total bitrate and entropy model|

---

EP3041233A1|2014-12-31|2016-07-06|Thomson Licensing|High frame rate-low frame rate transmission technique|

---

US9749646B2|2015-01-16|2017-08-29|Microsoft Technology Licensing, Llc|Encoding/decoding of high chroma resolution details|

---

CN107211193B|2015-02-07|2021-04-13|视觉波公司|Intelligent adaptive video streaming method and system driven by perception experience quality estimation|

---

CN104767999B|2015-04-22|2017-11-17|福州大学|A kind of HEVC Rate Controls model parameter more new algorithm based on distortion measurement|

---

US10735773B2|2015-06-04|2020-08-04|Apple Inc.|Video coding techniques for high quality coding of low motion content|

---

US9734409B2|2015-06-24|2017-08-15|Netflix, Inc.|Determining native resolutions of video sequences|

---

US10602153B2|2015-09-11|2020-03-24|Facebook, Inc.|Ultra-high video compression|

---

US10255667B2|2015-12-23|2019-04-09|Vmware, Inc.|Quantitative visual perception quality measurement for virtual desktops|

---

CN105868700A|2016-03-25|2016-08-17|哈尔滨工业大学深圳研究生院|Vehicle type recognition and tracking method and system based on monitoring video|

---

US20180063549A1|2016-08-24|2018-03-01|Ati Technologies Ulc|System and method for dynamically changing resolution based on content|

---

AU2017368324A1|2016-12-01|2019-05-30|Brightcove, Inc.|Optimization of encoding profiles for media streaming|

---

US10742708B2|2017-02-23|2020-08-11|Netflix, Inc.|Iterative techniques for generating multiple encoded versions of a media title|

---

US10897618B2|2017-02-23|2021-01-19|Netflix, Inc.|Techniques for positioning key frames within encoded video sequences|

---

US11153585B2|2017-02-23|2021-10-19|Netflix, Inc.|Optimizing encoding operations when generating encoded versions of a media title|

---

US10666992B2|2017-07-18|2020-05-26|Netflix, Inc.|Encoding techniques for optimizing distortion and bitrate|

---

US20190028529A1|2017-07-18|2019-01-24|Netflix, Inc.|Encoding techniques for optimizing distortion and bitrate|EP3345385A1|2015-09-04|2018-07-11|Telefonaktiebolaget LM Ericsson |Improved cloud dvr storage|

---

US10148989B2|2016-06-15|2018-12-04|Divx, Llc|Systems and methods for encoding video content|

---

US11153585B2|2017-02-23|2021-10-19|Netflix, Inc.|Optimizing encoding operations when generating encoded versions of a media title|

---

US10897618B2|2017-02-23|2021-01-19|Netflix, Inc.|Techniques for positioning key frames within encoded video sequences|

---

US11166034B2|2017-02-23|2021-11-02|Netflix, Inc.|Comparing video encoders/decoders using shot-based encoding and a perceptual visual quality metric|

---

WO2021034578A1|2019-08-16|2021-02-25|Netflix, Inc.|Comparing video encoders/decoders using shot-based encoding and a perceptual visual quality metric|

---

US10742708B2|2017-02-23|2020-08-11|Netflix, Inc.|Iterative techniques for generating multiple encoded versions of a media title|

---

US10666992B2|2017-07-18|2020-05-26|Netflix, Inc.|Encoding techniques for optimizing distortion and bitrate|

---

US10616590B1|2018-05-16|2020-04-07|Amazon Technologies, Inc.|Optimizing streaming video encoding profiles|

---

US10554975B1|2018-09-30|2020-02-04|Tencent America LLC|Method and apparatus for video coding|

---

US11128869B1|2018-10-22|2021-09-21|Bitmovin, Inc.|Video encoding based on customized bitrate table|

---

US10841356B2|2018-11-28|2020-11-17|Netflix, Inc.|Techniques for encoding a media title while constraining bitrate variations|

---

US10880354B2|2018-11-28|2020-12-29|Netflix, Inc.|Techniques for encoding a media title while constraining quality variations|

---

US10897654B1|2019-09-30|2021-01-19|Amazon Technologies, Inc.|Content delivery of live streams with event-adaptive encoding|

---

US11115697B1|2019-12-06|2021-09-07|Amazon Technologies, Inc.|Resolution-based manifest generator for adaptive bitrate video streaming|

---

US10958947B1|2020-03-12|2021-03-23|Amazon Technologies, Inc.|Content delivery of live streams with playback-conditions-adaptive encoding|

---

US20210329254A1|2020-04-16|2021-10-21|Alibaba Group Holding Limited|Bitrate control method for video coding|

法律状态:
2021-10-13| B350| Update of information on the portal [chapter 15.35 patent gazette]|

优先权:

---

申请号 | 申请日 | 专利标题

---

US201762462873P| true| 2017-02-23|2017-02-23|

---

US62/462,873|2017-02-23|

---

US201762534170P| true| 2017-07-18|2017-07-18|

---

US62/534,170|2017-07-18|

---

US201762550517P| true| 2017-08-25|2017-08-25|

---

US62/550,517|2017-08-25|

---

US15/902,976|US10917644B2|2017-02-23|2018-02-22|Iterative techniques for encoding video content|

---

US15/902,976|2018-02-22|

---

PCT/US2018/019576|WO2018156997A1|2017-02-23|2018-02-23|Iterative techniques for encoding video content|

---